Papers with global representation
Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders (2021.acl-long)
Copied to clipboard
| Challenge: | End-to-end Speech Translation (E2E ST) encoders lack global context representation, whereas MT encoder lacks it. |
| Approach: | They propose a Stacked Acoustic-and-Textual Encoding method for speech translation . they propose an adaptor module to alleviate representation inconsistency . |
| Outcome: | The proposed method achieves state-of-the-art BLEU scores of 18.3 and 25.2 on two ST tasks. |
GCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling (P19-1)
Copied to clipboard
| Challenge: | Existing systems for sequence labeling are limited by shallow connections between consecutive hidden states and insufficient modeling of global information. |
| Approach: | They propose a global context enhanced deep transition architecture for sequence labeling . they deepen the state transition path at each position in a sentence and assign tokens with global representations . |
| Outcome: | The proposed architecture outperforms the best reported results on two standard sequence labeling tasks. |
FiE: Building a Global Probability Space by Leveraging Early Fusion in Encoder for Open-Domain Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | generative models tend to be larger than extractive models due to the need for a decoder, run slower during inference due to auto-regressive decoded beam search, and their generated output suffers from hallucinations. |
| Approach: | They propose to extend transformer encoders with the ability to fuse information from multiple passages to provide cross-sample attention over all tokens across samples. |
| Outcome: | The proposed method outperforms the current state-of-the-art method by 2.5 Exact Match score on the Natural Question dataset while using only 25% of parameters and 35% of the latency during inference. |
A State-independent and Time-evolving Network for Early Rumor Detection in Social Media (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to rumor detection ignored dynamical evolution of an event and failed to capture its unique features in different states. |
| Approach: | They propose a state-independent and time-evolving Network (STN) for rumor detection based on fine-grained event state detection and segmentation. |
| Outcome: | The proposed framework can significantly improve the rumor detection accuracy in comparison with some strong baseline systems. |
TARE: Lightweight Token-Aware Representation Editing for Fine-tuning Transformer-like Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing PEFT methods can be costly and underfit token-level contexts. |
| Approach: | They propose a PEFT method that performs fine-grained, token-specific edits with a small additional inference overhead and minimal tuning. |
| Outcome: | The proposed method outperforms state-of-the-art methods in 8 tasks and GLUE with a minimal tuning overhead and inference overhead. |
Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interaction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Emotion recognition is a crucial task for human conversation understanding . multimodal data, e.g., language, voice, and facial expressions, add complexity to the task. |
| Approach: | They propose a relational temporal Graph Neural Network with Auxiliary Cross-Modality Interaction framework that captures conversation-level cross-modality interactions and utterance-level temporal dependencies with modality-specific manner for conversation understanding. |
| Outcome: | The proposed framework captures conversation-level cross-modality interactions and utterance-level temporal dependencies with the modality-specific manner for conversation understanding. |
Gated Differentiable Working Memory for Long-Context Language Modeling (2026.acl-long)
Copied to clipboard
Lingrui Mei, Shenghua Liu, Yiwei Wang, Yuyao Ge, Baolong Bi, Jiayu Yao, Jun Wan, Ziling Yin, Jiafeng Guo, Xueqi Cheng
| Challenge: | Long contexts break transformers, attention scores dilute, model cannot adapt to novel patterns at inference time. |
| Approach: | They propose a framework that gates the memory consolidation process by estimating Contextual Utility . they propose GDWM to maintain a form of working memory with constant contexts . |
| Outcome: | The proposed framework achieves comparable or superior performance on sparse-information tasks with 4 fewer gradient steps compared to uniform baselines. |
Reinforcement Learning–Guided Adaptive Tuning for Out-of-Distribution Harmful Text Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for testing harmful information on social media rely on fixed parameters that fail to handle substantial semantic discrepancies . RLAT can be used to adapt to semantic variations while preventing overfitting from continuous tuning. |
| Approach: | They propose a reinforcement learning-guided adaptive tuning method for harmful text detection that optimizes consistency loss and applies word-level attention constraints to reduce over-reliance on local words. |
| Outcome: | The proposed method outperforms state-of-the-art models in cross-platform and cross-temporal scenarios across multiple public datasets. |